Predecessor Features

Any reinforcement learning system must be able to identify which past events contributed to observed outcomes, a problem known as credit assignment. A common solution to this problem is to use an eligibility trace to assign credit to recency-weighted set of experienced events. However, in many realistic tasks, the set of recently experienced events are only one of the many possible action events that could have preceded the current outcome. This suggests that reinforcement learning can be made more efficient by allowing credit assignment to any viable preceding state, rather than only those most recently experienced. Accordingly, we examine ``Predecessor Features'', the fully bootstrapped version of van Hasselt's ``Expected Trace'', an algorithm that achieves this richer form of credit assignment. By maintaining a representation that approximates the expected sum of past occupancies, this algorithm allows temporal difference (TD) errors to be propagated accurately to a larger number of predecessor states than conventional methods, greatly improving learning speed. The algorithm can also be naturally extended from tabular state representation to feature representations allowing for increased performance on a wide range of environments. We demonstrate several use cases for Predecessor Features and compare its performance with other approaches.Comment: Accepted to RLDM 202

Emergence and reconfiguration of modular structure for synaptic neural networks during continual familiarity detection

While advances in artificial intelligence and neuroscience have enabled the emergence of neural networks capable of learning a wide variety of tasks, our understanding of the temporal dynamics of these networks remains limited. Here, we study the temporal dynamics during learning of Hebbian Feedforward (HebbFF) neural networks in tasks of continual familiarity detection. Drawing inspiration from the field of network neuroscience, we examine the network's dynamic reconfiguration, focusing on how network modules evolve throughout learning. Through a comprehensive assessment involving metrics like network accuracy, modular flexibility, and distribution entropy across diverse learning modes, our approach reveals various previously unknown patterns of network reconfiguration. In particular, we find that the emergence of network modularity is a salient predictor of performance, and that modularization strengthens with increasing flexibility throughout learning. These insights not only elucidate the nuanced interplay of network modularity, accuracy, and learning dynamics but also bridge our understanding of learning in artificial and biological realms

Adaptation decorrelates shape representations

Perception and neural responses are modulated by sensory history. Visual adaptation, an example of such an effect, has been hypothesized to improve stimulus discrimination by decorrelating responses across a set of neural units. While a central theoretical model, behavioral and neural evidence for this theory is limited and inconclusive. Here, we use a parametric 3D shape-space to test whether adaptation decorrelates shape representations in humans. In a behavioral experiment with 20 subjects, we find that adaptation to a shape class improves discrimination of subsequently presented stimuli with similar features. In a BOLD fMRI experiment with 10 subjects, we observe that adaptation to a shape class decorrelates the multivariate representations of subsequently presented stimuli with similar features in object-selective cortex. These results support the long-standing proposal that adaptation improves perceptual discrimination and decorrelates neural representations, offering insights into potential underlying mechanisms

Functional Network Dynamics of the Language System

During linguistic processing, a set of brain regions on the lateral surfaces of the left frontal, temporal, and parietal cortices exhibit robust responses. These areas display highly correlated activity while a subject rests or performs a naturalistic language comprehension task, suggesting that they form an integrated functional system. Evidence suggests that this system is spatially and functionally distinct from other systems that support high-level cognition in humans. Yet, how different regions within this system might be recruited dynamically during task performance is not well understood. Here we use network methods, applied to fMRI data collected from 22 human subjects performing a language comprehension task, to reveal the dynamic nature of the language system. We observe the presence of a stable core of brain regions, predominantly located in the left hemisphere, that consistently coactivate with one another. We also observe the presence of a more flexible periphery of brain regions, predominantly located in the right hemisphere, that coactivate with different regions at different times. However, the language functional ROIs in the angular gyrus and the anterior temporal lobe were notable exceptions to this trend. By highlighting the temporal dimension of language processing, these results suggest a trade-off between a region's specialization and its capacity for flexible network reconfiguration

The validity and reliability of the CAMDEX-DS for assessing dementia in adults with Down syndrome in Brazil.

OBJECTIVE: Alzheimer's disease occurs at a higher prevalence and an earlier age in individuals with Down syndrome (DS) than typically developing individuals. However, diagnosing dementia in individuals with intellectual disability remains a challenge due to pre-existing cognitive deficits. The aim of this study was to investigate the validity and reliability of the Brazilian version of the Cambridge Examination for Mental Disorders of Older People with Down's syndrome and Others with Intellectual Disabilities (CAMDEX-DS) for individuals with DS. METHODS: Two psychiatrists, working independently, evaluated 92 adults with DS ≥ 30 years of age. The concurrent validity of the CAMDEX-DS was analyzed in relation to the gold standard of established international criteria. In a subgroup of 20 subjects, the concurrent validity of the CAMDEX-DS was analyzed in relation to an independent objective assessment of cognitive decline over three years. We analyzed the inter-rater reliability of cognitive assessment. RESULTS: The diagnostic accuracy of the CAMDEX-DS compared to the gold standard was 96.7%. CAMDEX-DS-based diagnosis was considered consistent with cognitive decline. The probability of a participant with dementia having cognitive decline was 83%. Inter-rater reliability for the participant assessment was good, with a kappa of > 0.8 for 93% of the CAMDEX-DS items. CONCLUSION: The CAMDEX-DS can be considered the first valid and reliable instrument for evaluating dementia in adults with DS in Brazil. Its use in such individuals could improve clinical practice and research

The validity and reliability of the CAMDEX-DS for assessing dementia in adults with Down syndrome in Brazil.

OBJECTIVE: Alzheimer's disease occurs at a higher prevalence and an earlier age in individuals with Down syndrome (DS) than typically developing individuals. However, diagnosing dementia in individuals with intellectual disability remains a challenge due to pre-existing cognitive deficits. The aim of this study was to investigate the validity and reliability of the Brazilian version of the Cambridge Examination for Mental Disorders of Older People with Down's syndrome and Others with Intellectual Disabilities (CAMDEX-DS) for individuals with DS. METHODS: Two psychiatrists, working independently, evaluated 92 adults with DS ≥ 30 years of age. The concurrent validity of the CAMDEX-DS was analyzed in relation to the gold standard of established international criteria. In a subgroup of 20 subjects, the concurrent validity of the CAMDEX-DS was analyzed in relation to an independent objective assessment of cognitive decline over three years. We analyzed the inter-rater reliability of cognitive assessment. RESULTS: The diagnostic accuracy of the CAMDEX-DS compared to the gold standard was 96.7%. CAMDEX-DS-based diagnosis was considered consistent with cognitive decline. The probability of a participant with dementia having cognitive decline was 83%. Inter-rater reliability for the participant assessment was good, with a kappa of > 0.8 for 93% of the CAMDEX-DS items. CONCLUSION: The CAMDEX-DS can be considered the first valid and reliable instrument for evaluating dementia in adults with DS in Brazil. Its use in such individuals could improve clinical practice and research